Head mounted display

ABSTRACT

A head mounted display (HMD) according to the present invention is capable of controlling various functions on the basis of movement of a main body thereof. In addition, in order to control the various functions, the HMD can set a moving area where the main body is to be moved, and display alert information when the main body deviates from the moving area. More specifically, an HMD according to one embodiment of the present invention comprises: a main body configured to be worn on a user&#39;s head; a display unit, installed on one area of the main body, for outputting display screen information; and a control unit for controlling the display unit in a way that, in a taken image including an image of a preset object, the screen information is changed on the basis of detection of a deformation in the image, wherein the control unit controls the display unit in a way that, if at least one portion of the image is not detected in the taken image, alert information related to the movement of the main body is displayed together on the screen information.

TECHNICAL FIELD

The present invention relates to a head mounted display (hereinafter,referred to as ‘HMD’) capable of controlling functions by a movement ofa main body.

BACKGROUND ART

A head-mounted display (HMD) refers to various image display devicessuch as glasses that are worn on a user's head to allow the user to viewimages (contents). Various wearable computers have been developeddepending on the trend of weight reduction and size reduction of digitaldevices, and the HMD is also widely used. The HMD may be combined withan augmented reality technology, an N-screen technology, etc., beyond asimple display function, to provide various conveniences to the user.

Recently, with the increase in the use of the HMD, methods of performingvarious functions according to a posture of the HMD are being activelydeveloped. Here, the posture of the HMD is a concept including a placedstate of the HMD (a worn state on the user's head), a tilted degree ofthe HMD, a direction that the HMD faces, a position of the HMD, arotation of the HMD, a movement of the HMD, and the like.

In the related art, an inclination or a movement of the HMD has beendetected by using sensors mounted on the HMD or the like, and theposture of the HMD has been determined based on the detected result.However, in such a method, it is difficult to detect the movement of theHMD when the HMD moves in a horizontal direction without being tilted(for example, when the HMD moves in parallel to front, rear, right andleft without being tilted).

Accordingly, when a taken (captured) image of a specific subject ischanged according to the movement (motion) of the main body, a method ofanalyzing the image change of the specific subject included in the takenimage to recognize the movement of the main body may be implemented.This method may enable a determination of even a movement of the HMD ina horizontal direction without being tilted as well as a tilted movementof the HMD. Accordingly, various functions can be more effectivelycontrolled according to the movement of the main body.

However, this method fails to precisely recognize the movement of themain body when the specific subject is not located within a specificviewing angle range (or a specific range of field of view) any more dueto the movement of the main body, which results from particularity thata camera forming the taken image is capable of capturing only thespecific viewing angle range.

DISCLOSURE OF THE INVENTION

Accordingly, an aspect of the present invention is to provide a headmounted display (HMD), capable of outputting alert information notifyinga situation that a preset object for analyzing a movement of a main bodyis out of a viewing angle range of a camera due to the movement of themain body, and a method of controlling the same.

To achieve these and other advantages and in accordance with the purposeof this specification, as embodied and broadly described herein, thereis provided a head mounted display, including a main body formed to bewearable on a head portion of a user, a display unit mounted on one areaof the main body and configured to output screen information, and acontroller configured to control the display unit to change the screeninformation based on a detection of deformation of an image of thepreset object in a captured image including the image of the presetobject, wherein the controller controls the display unit so that alertinformation related to the movement of the main body is output togetherwith the screen information when at least part of the image is notdetected in the captured image.

In one embodiment, the head mounted display may further include a cameramounted on the main body. The preset object may be disposed outside theHMD, and the controller may acquire the captured image including theimage of the preset object from the camera.

In one embodiment, the head mounted display may further include awireless communication unit, and the preset object may be mounted on atleast part of the main body. The controller may control the wirelesscommunication unit to receive the captured image including the image ofthe preset object from a camera located outside the HMD.

In one embodiment, the preset object may be a plurality of lightemitting elements each emitting preset light.

In one embodiment, the controller may control the display unit to changean output of the alert information based on a moving speed of the mainbody determined from the captured image.

In one embodiment, the controller may determine the moving speed of themain body using a degree of change of at least part of the image for apreset reference time when the at least part of the image is notdetected in the captured image.

In one embodiment, the controller may control the display unit such thatthe alert information overlaps the screen information to correspond to afirst degree of change when the degree of change of the at least part ofthe image is the first degree of change, and control the display unit todisplay recommendation information for another function, different froma function corresponding to the screen information, on the alertinformation when an overlap rate of the alert information with thescreen information corresponds to a preset specific rate.

In one embodiment, the controller may execute the another functionincluded in the recommendation information when the entire image is notdetected in the captured image in a state where the recommendationinformation for the another function is displayed on the alertinformation.

In one embodiment, the controller may terminate an execution of thefunction corresponding to the screen information when the anotherfunction is executed, and control the display unit to output anexecution screen according to the execution of the another function.

In one embodiment, the head mounted display may further include asensing unit configured to sense a preset gesture. The controller mayexecute the another function while maintaining the execution of thefunction corresponding to the screen information when the preset gestureis detected by the sensing unit in a state where the recommendationinformation is output, and control the display unit to output anexecution screen according to the execution of the another functiontogether with the screen information.

In one embodiment, the head mounted display may further a sensing unitconfigured to sense a preset gesture. The controller may execute theanother function included in the recommendation information when thepreset gesture is detected by the sensing unit in a state where therecommendation information is displayed, and control the display unit tochange the screen information to the execution screen according to theexecution of the another function.

In one embodiment, the controller may control the display unit to changethe execution screen according to the execution of the another functionback to the screen information when the entire image is detected in thecaptured image in a state where the screen information has changed tothe execution screen according to the execution of the another function.

In one embodiment, when the degree of change of the at least part of theimage is a second degree of change, the controller may control thedisplay unit so that the alert information includes informationindicating that the main body is out of a moving area of the main bodyin which the main body is allowed to control the screen information.

In one embodiment, when a specific part of the image is not detected inthe captured image, the controller may control the display unit tooutput the alert information at a position, which corresponds to aposition of the specific part with respect to the image on the displayunit.

In one embodiment, when a movement of a specific subject with respect tothe main body is detected from the captured image and at least part ofthe image is not detected based on the movement of the specific subject,the controller may control the display unit such that the alertinformation includes notification information indicating presence of thespecific subject.

To achieve these and other advantages and in accordance with the purposeof this specification, as embodied and broadly described herein, thereis provided a method for controlling a head mounted display (HMD), themethod including outputting screen information on a display unit mountedon one area of a main body, detecting a deformation of an image of apreset object in a captured image including the image of the presetobject, changing the screen information based on the detection of thedeformation of the image, and outputting alert information related to amovement of the main body together with the screen information when atleast part of the image is not detected in the captured image.

In one embodiment, the preset object may be disposed outside the HMD,and the captured image including the image of the preset object may beacquired from a camera mounted on the main body.

In one embodiment, the preset object may be mounted on at least part ofthe main body, and the captured image including the image of the presetobject may be received from a camera located outside the HMD through awireless communication unit performing wireless communication with thecamera.

In one embodiment, the alert information may be differently displayedbased on a degree of change of the at least part of the image for apreset reference time when the at least part of the image is notdetected in the captured image.

EFFECTS OF THE INVENTION

According to the present invention, a movement of a main body of a headmounted display (HMD) can be recognized by using a captured image withrespect to a preset object, thereby accurately detecting variousmovements of the main body, and controlling various functions using thedetected movements.

In addition, alert information can be output when capturing of at leastpart of the preset object is failed due to the movement of the mainbody, which may allow the user to recognize a moved state of the mainbody.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a head mounted display (HMD) in accordancewith the present invention.

FIGS. 2A and 2B are conceptual views of an HMD according to oneembodiment, viewed in one direction.

FIG. 3 is a flowchart illustrating a control method in accordance withthe present invention.

FIGS. 4A, 4B, and 4C are representative views illustrating a method ofcontrolling an HMD in accordance with one embodiment of the presentinvention.

FIGS. 5A, 5B and 5C are views illustrating embodiments related tovarious output forms of alert information.

FIGS. 6A and 6B are views illustrating an embodiment for controlling anoutput (display) of alert information based on a moving speed of a mainbody.

FIGS. 7A and 7B are views illustrating an embodiment related to anexecution of a function related to specific information when the mainbody is moved in a state where the specific information is additionallydisplayed together with alert information.

FIGS. 8A and 8B are views illustrating an embodiment related tomultitasking of a function corresponding to screen information andanother function included in recommendation information.

FIGS. 9A and 9B are views illustrating an embodiment in which an outputstate of a display unit is controlled based on a movement of a main bodyin a state where alert information overlaps entire screen information.

FIG. 10 is a view illustrating an embodiment in which an output positionof alert information is changed based on a direction in which the mainbody moves out of a moving area.

FIG. 11 is a view illustrating an embodiment of outputting informationinforming a movement of a specific subject (object) with respect to amain body.

BEST MODE OF THE INVENTION

Description will now be given in detail according to exemplaryembodiments disclosed herein, with reference to the accompanyingdrawings. For the sake of brief description with reference to thedrawings, the same or equivalent components may be provided with thesame or similar reference numbers, and description thereof will not berepeated. In general, a suffix such as “module” and “unit” may be usedto refer to elements or components. Use of such a suffix herein ismerely intended to facilitate description of the specification, and thesuffix itself is not intended to give any special meaning or function.In describing the present disclosure, if a detailed explanation for arelated known function or construction is considered to unnecessarilydivert the gist of the present disclosure, such explanation has beenomitted but would be understood by those skilled in the art. Theaccompanying drawings are used to help easily understand the technicalidea of the present disclosure and it should be understood that the ideaof the present disclosure is not limited by the accompanying drawings.The idea of the present disclosure should be construed to extend to anyalterations, equivalents and substitutes besides the accompanyingdrawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element can be connected with theanother element or intervening elements may also be present. Incontrast, when an element is referred to as being “directly connectedwith” another element, there are no intervening elements present.

A singular representation may include a plural representation unless itrepresents a definitely different meaning from the context.

Terms such as “include” or “has” are used herein and should beunderstood that they are intended to indicate an existence of severalcomponents, functions or steps, disclosed in the specification, and itis also understood that greater or fewer components, functions, or stepsmay likewise be utilized.

The HMD described in this specification may include a wearable device(e.g., a smart glass) and the like.

However, it will be easily understood by those skilled in the art thatthe configuration according to the embodiments described herein, unlessotherwise limited to a case of being applicable only to the HMD, may beapplied to mobile terminals, such as mobile phones, smart phones, laptopcomputers, digital broadcasting terminals, personal digital assistants(PDAs), personal digital assistants (PMPs), navigation systems, slatePCs, tablet PCs, ultrabooks, smartwatches, and the like.

Referring to FIG. 1, FIG. 1 is a block diagram of a head mounted display(HMD) in accordance with the present invention.

The HMD 100 may be shown having components such as a wirelesscommunication unit 110, an input unit 120, a sensing unit 140, an outputunit 150, an interface unit 160, a memory 170, a controller (or controlunit) 180, and a power supply unit 190. It is understood thatimplementing all of the illustrated components is not a requirement, andthat greater or fewer components may alternatively be implemented.

More specifically, among those components, the wireless communicationunit 110 may include at least one module for allowing wirelesscommunication between the HMD 100 and a wireless communication system,between the HMD 100 and another HMD, between the HMD 100 and a mobile orstationary terminal, between the HMD 100 and a control device, betweenthe HMD 100 and a camera externally installed to perform wirelesscommunication, or between the HMD 100 and an external server.

Further, the wireless communication unit 110 may typically include oneor more modules which connect the HMD 100 to one or more networks.

The wireless communication unit 110 may include at least one of abroadcast receiving module 111, a mobile communication module 112, awireless Internet module 113, a short-range communication module 114,and a location information module 115.

The input unit 120 may include a camera 121 or an image input unit forobtaining images or video, a microphone 122, which is one type of audioinput device for inputting an audio signal, and a user input unit 123(for example, a touch key, a mechanical key, and the like) for allowinga user to input information. Data (for example, audio, video, image, andthe like) may be obtained by the input unit 120 and may be analyzed andprocessed according to user commands.

The sensing unit 140 may typically be implemented using one or moresensors configured to sense internal information of the HMD, thesurrounding environment of the HMD, user information, and the like. Forexample, the sensing unit 140 may include at least one of a proximitysensor 141, an illumination sensor 142, a touch sensor, an accelerationsensor, a magnetic sensor, a G-sensor, a gyroscope sensor, a motionsensor, an RGB sensor, an infrared (IR) sensor, a finger scan sensor, aultrasonic sensor, an optical sensor (for example, camera 121), amicrophone 122, a battery gauge, an environment sensor (for example, abarometer, a hygrometer, a thermometer, a radiation detection sensor, athermal sensor, and a gas sensor, among others), and a chemical sensor(for example, an electronic nose, a health care sensor, a biometricsensor, and the like). The HMD disclosed herein may be configured toutilize information obtained from one or more sensors of the sensingunit 140, and combinations thereof.

The output unit 150 may typically be configured to output various typesof information, such as audio, video, tactile output, and the like. Theoutput unit 150 may be shown having at least one of a display unit 151,an audio output module 152, a haptic module 153, and an optical outputmodule 154.

The interface unit 160 serves as an interface with various types ofexternal devices that are coupled to the HMD 100. The interface unit160, for example, may include any of wired or wireless ports, externalpower supply ports, wired or wireless data ports, memory card ports,ports for connecting a device having an identification module, audioinput/output (I/O) ports, video I/O ports, earphone ports, and the like.In some cases, the HMD 100 may perform assorted control functionsassociated with a connected external device, in response to the externaldevice being connected to the interface unit 160.

The memory 170 is typically implemented to store data to support variousfunctions or features of the HMD 100. For instance, the memory 170 maybe configured to store application programs executed in the HMD 100,data or instructions for operations of the HMD 100, and the like. Someof these application programs may be downloaded from an external servervia wireless communication. Other application programs may be installedwithin the HMD 100 at time of manufacturing or shipping, which istypically the case for basic functions of the HMD 100 (for example,receiving a call, placing a call, receiving a message, sending amessage, and the like). Application programs may be stored in the memory170, installed in the HMD 100, and executed by the controller 180 toperform an operation (or function) for the HMD 100.

The controller 180 typically functions to control an overall operationof the HMD 100, in addition to the operations associated with theapplication programs. The controller 180 may provide or processinformation or functions appropriate for a user by processing signals,data, information and the like, which are input or output by theaforementioned various components, or activating application programsstored in the memory 170.

Also, the controller 180 may control at least some of the componentsillustrated in FIG. 1A, to execute an application program that have beenstored in the memory 170. In addition, the controller 180 may control atleast two of those components included in the HMD to activate theapplication program.

The power supply unit 190 may be configured to receive external power orprovide internal power in order to supply appropriate power required foroperating elements and components included in the HMD 100. The powersupply unit 190 may include a battery, and the battery may be configuredto be embedded in the terminal body, or configured to be detachable fromthe terminal body.

At least part of the components may cooperatively operate to implementan operation, a control or a control method of an HMD according tovarious embodiments disclosed herein. Also, the operation, the controlor the control method of the HMD may be implemented on the HMD by anactivation of at least one application program stored in the memory 170.

Hereinafter, description will be given in more detail of theaforementioned components with reference to FIG. 1A, prior to describingvarious embodiments implemented through the HMD 100.

First, regarding the wireless communication unit 110, the broadcastreceiving module 111 is typically configured to receive a broadcastsignal and/or broadcast associated information from an externalbroadcast managing entity via a broadcast channel. The broadcast channelmay include a satellite channel, a terrestrial channel, or both. In someembodiments, two or more broadcast receiving modules may be provided inthe HMD 100 to facilitate simultaneous reception of two or morebroadcast channels, or to support switching among broadcast channels.

The broadcast management server may be a server that generates andtransmits a broadcast signal and/or broadcast associated information ora server that receives a previously generated broadcast signal and/orbroadcast associated information and transmits the same to a device(e.g., a control device, a terminal, etc.) which is connected to the HMDto control the HMD. The broadcast signal may include a TV broadcastsignal, a radio broadcast signal, a data broadcast signal, and the like.Also, the broadcast signal may further include a broadcast signal in aform that a TV or radio broadcast signal is combined with a databroadcast signal.

The broadcast signal may be encrypted by at least one of technicalstandards (or broadcasting methods, e.g., ISO, IEC, DVB, ATSC, etc.) fortransmitting and receiving a digital broadcast signal. The broadcastreceiving module 111 may receive the digital broadcast signal using amethod suitable for a technical standard selected from those technicalstandards.

Examples of the broadcast associated information may include informationassociated with a broadcast channel, a broadcast program, a broadcastservice provider, and the like. The broadcast associated information maybe provided via a mobile communication network. In this case, thebroadcast associated information may be received by the mobilecommunication module 112.

The broadcast associated information may be implemented in variousformats. For instance, broadcast associated information may includeElectronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB),Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld(DVB-H), and the like. The broadcast signal and/or the broadcast relatedinformation received through the broadcast receiving module 111 may bestored in the memory 170.

The mobile communication module 112 can transmit and/or receive wirelesssignals to and from one or more network entities. Typical examples of anetwork entity include a base station, an external mobile terminal, aserver, and the like. Such network entities form part of a mobilecommunication network, which is constructed according to technicalstandards or communication methods for mobile communications (forexample, Global System for Mobile Communication (GSM), Code DivisionMulti Access (CDMA), CDMA2000(Code Division Multi Access 2000),EV-DO(Enhanced Voice-Data Optimized or Enhanced Voice-Data Only),Wideband CDMA (WCDMA), High Speed Downlink Packet access (HSDPA), HSUPA(High Speed Uplink Packet Access), Long Term Evolution (LTE), LTE-A(Long Term Evolution-Advanced), and the like).

The wireless signal may include various types of data depending on avoice call signal, a video call signal, or a text/multimedia messagetransmission/reception.

The wireless Internet module 113 refers to a module for wirelessInternet access. This module may be internally or externally coupled tothe electronic device 100. The wireless Internet module 113 may transmitand/or receive wireless signals via communication networks according towireless Internet technologies.

Examples of such wireless Internet access include Wireless LAN (WLAN),Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance(DLNA), Wireless Broadband (WiBro), Worldwide Interoperability forMicrowave Access (WiMAX), High Speed Downlink Packet Access (HSDPA),High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE),LTE-advanced (LTE-A) and the like. The wireless Internet module 113 maytransmit/receive data according to one or more of such wireless Internettechnologies, and other Internet technologies as well.

When the wireless Internet access is implemented according to, forexample, WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A and the like,as part of a mobile communication network, the wireless Internet module113 performs such wireless Internet access. As such, the Internet module113 may cooperate with, or function as, the mobile communication module112.

The short-range communication module 114 is configured to facilitateshort-range communications. Suitable technologies for implementing suchshort-range communications include BLUETOOTH™, Radio FrequencyIDentification (RFID), Infrared Data Association (IrDA), Ultra-WideBand(UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity(Wi-Fi), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), andthe like. The short-range communication module 114 in general supports,through wireless area networks, wireless communications between the HMD100 and a wireless communication system, between the HMD 100 and anotherHMD, between the HMD 100 and a mobile or stationary terminal, betweenthe HMD 100 and a control device, between the HMD 100 and a cameraexternally installed to perform wireless communication, or between theHMD 100 and an external server. One example of the wireless areanetworks is a wireless personal area networks.

Here, the HMD may be a device (e.g., a mobile phone, a smart phone, asmartwatch, a laptop computer, a controller, etc.) which can exchangedata with (or cooperative with) the HMD 100. The short-rangecommunication module 114 may sense (or recognize) a device, which canperform communication with the HMD 100, in the vicinity of the HMD 100.In addition, when the sensed device is a device which is authenticatedto communicate with the HMD 100, the controller 180, for example, maycause transmission of at least part of data processed in the HMD 100 tothe device or at least part of data processed in the device to the HMD100 via the short-range communication module 114.

Hence, a user of the HMD 100 may use the data processed in the devicethrough the HMD 100. For example, when a call is received in the device,the user may answer the call using the HMD 100. Also, when a message isreceived in the device, the user may check the received message usingthe HMD 100.

The location information module 115 is a module for acquiring a position(or a current position) of the HMD 100. As an example, the locationinformation module 115 includes a Global Position System (GPS) module ora Wi-Fi module. For example, when the HMD uses a GPS module, a positionof the HMD may be acquired using a signal sent from a GPS satellite. Asanother example, when the HMD uses the Wi-Fi module, a position of theHMD may be acquired based on information related to a wireless accesspoint (AP) which transmits or receives a wireless signal to or from theWi-Fi module. If desired, the location information module 115 mayalternatively or additionally function with any of the other modules ofthe wireless communication unit 110 to obtain data related to theposition of the HMD. The location information module 115 is a moduleused for acquiring the position (or the current position) of the HMD,and may not be limited to a module for directly calculating or acquiringthe position of the HMD.

Next, the input unit 120 is configured to permit various types of inputsto the HMD 100. Examples of such inputs include image information (orsignal), audio information (or signal), data or various informationinput by a user, and may be provided with one or a plurality of cameras121. Such cameras 121 may process image frames of still pictures orvideo obtained by image sensors in a video or image capture mode. Theprocessed image frames can be displayed on the display unit 151 orstored in memory 170. Meanwhile, the cameras 121 provided in the HMD 100may be arranged in a matrix configuration to permit a plurality ofimages having various angles or focal points to be input to the HMD 100.Also, the cameras 121 may be located in a stereoscopic arrangement toacquire left and right images for implementing a stereoscopic image.

The microphone 122 processes an external audio signal into electricaudio (sound) data. The processed audio data may be processed in variousmanners according to a function being executed in the HMD 100. Ifdesired, the microphone 122 may include assorted noise removingalgorithms to remove unwanted noise generated in the course of receivingthe external audio signal.

The user input unit 123 is a component that permits input by a user.Such user input may enable the controller 180 to control an operation ofthe HMD 100. The user input unit 123 may include one or more of amechanical input element (for example, a mechanical key, a buttonlocated on a front and/or rear surface or a side surface of the HMD 100,a dome switch, a jog wheel, a jog switch, and the like), or atouch-sensitive input element, among others. As one example, thetouch-sensitive input element may be at least one of a touch pad and atouch panel.

The sensing unit 140 is generally configured to sense one or more ofinternal information of the HMD, surrounding environment information ofthe HMD, user information, or the like, and generate a correspondingsensing signal. The controller 180 generally cooperates with the sendingunit 140 to control operation of the HMD 100 or execute data processing,a function or an operation associated with an application programinstalled in the HMD 100 based on the sensing signal. The sensing unit140 may be implemented using any of a variety of sensors, some of whichwill now be described in more detail.

The proximity sensor 141 refers to a sensor to sense presence or absenceof an object approaching a surface, or an object located near a surface,by using an electromagnetic field, infrared rays, or the like without amechanical contact. The proximity sensor 141 may be arranged at an innerregion of the HMD covered by the touch screen, or near the touch screen.

The proximity sensor 141, for example, may include any of a transmissivetype photoelectric sensor, a direct reflective type photoelectricsensor, a mirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared rays proximity sensor, andthe like. When the touch-sensitive input element of the user input unit123 is implemented as a capacitance type, the proximity sensor 141 maysense proximity of a pointer having conductivity by changes of anelectromagnetic field, which is responsive to an approach of thepointer. In this case, the user input unit 123 itself may also becategorized as a proximity sensor.

The term “proximity touch” will often be referred to herein to denote amotion (action) that the pointer is closely located above the user inputunit 123 without being in contact with the user input unit 123. The term“contact touch” will often be referred to herein to denote a motion(action) that the pointer is actually brought into contact with the userinput unit 123. For the position corresponding to the proximity touch ofthe pointer relative to the user input unit 123, such position willcorrespond to a position where the pointer is perpendicular to the userinput unit. The proximity sensor 141 may sense proximity touch, andproximity touch patterns (for example, distance, direction, speed, time,position, moving status, and the like). In general, the controller 180processes data corresponding to proximity touches and proximity touchpatterns sensed by the proximity sensor 141, and cause output of visualinformation on the display unit 151. In addition, the controller 180 maycontrol the HMD 100 to execute different operations or process differentdata (or information) according to whether a touch with respect to thesame point on the user input unit 123 is either a proximity touch or acontact touch.

A touch sensor senses a touch (or a touch input) applied to the userinput unit 123 using at least one of a variety of touch methods.Examples of such touch methods include a resistive type, a capacitivetype, an infrared type, an ultrasonic type, and a magnetic field type,among others.

As one example, the touch sensor may be configured to convert changes ofpressure applied to a specific part of the user input unit 123, orconvert capacitance occurring at a specific part of the user input unit123, into electric input signals. The touch sensor may also beconfigured to sense a touch position, a touch area, touch pressure, andtouch capacitance when a touch object touches the user input unit 123. Atouch object is generally used to apply a touch input to the touchsensor. Examples of typical touch objects include a finger, a touch pen,a stylus pen, a pointer, or the like.

When a touch input is sensed by a touch sensor, corresponding signalsmay be transmitted to a touch controller. The touch controller mayprocess the received signals, and then transmit corresponding data tothe controller 180. Accordingly, the controller 180 may sense whichregion of the user input unit 123 has been touched. Here, the touchcontroller may be a component separate from the controller 180, thecontroller 180, and combinations thereof.

Meanwhile, the controller 180 may execute the same or different controlsaccording to a type of touch object that touches the user input unit123. Whether to execute the same or different control according to atype of a touch object may be decided based on a current operating stateof the HMD 100 or a currently executed application program, for example.

The touch sensor and the proximity sensor may be implementedindividually, or in combination, to sense various types of touches. Suchtouches includes a short (or tap) touch, a long touch, a multi-touch, adrag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipetouch, a hovering touch, and the like.

If desired, an ultrasonic sensor may be implemented to recognizelocation information relating to a touch object using ultrasonic waves.The controller 180, for example, may calculate a position of a wavegeneration source based on information sensed by an illumination sensorand a plurality of ultrasonic sensors. Since light is much faster thanultrasonic waves, the time for which the light reaches the opticalsensor is much shorter than the time for which the ultrasonic wavereaches the ultrasonic sensor. The position of the wave generationsource may be calculated using this fact. For instance, the position ofthe wave generation source may be calculated using the time differencefrom the time that the ultrasonic wave reaches the sensor based on thelight as a reference signal.

The camera 121, which has been depicted as a component of the input unit120, typically includes at least one a camera sensor (CCD, CMOS etc.), aphoto sensor (or image sensors), and a laser sensor.

Implementing the camera 121 with a laser sensor may allow detection of atouch of a physical object with respect to a 3D stereoscopic image. Thephoto sensor may be laminated on, or overlapped with, the displaydevice. The photo sensor may be configured to scan movement of thephysical object in proximity to the touch screen. In more detail, thephoto sensor may include photo diodes and transistors (TRs) at rows andcolumns to scan content received at the photo sensor using an electricalsignal which changes according to the quantity of applied light. Namely,the photo sensor may calculate the coordinates of the physical objectaccording to variation of light to thus obtain location information ofthe physical object.

The display unit 151 is generally configured to output informationprocessed in the HMD 100. For example, the display unit 151 may displayexecution screen information of an application program executing at theHMD 100 or user interface (UI) and graphic user interface (GUI)information in response to the execution screen information.

Also, the display unit 151 may be implemented as a stereoscopic displayunit for displaying stereoscopic images.

A typical stereoscopic display unit may employ a stereoscopic displayscheme such as a stereoscopic scheme (a glass scheme), anauto-stereoscopic scheme (glassless scheme), a projection scheme(holographic scheme), or the like.

In general, a 3D stereoscopic image is comprised of a left image (a lefteye image) and a right image (a right eye image). According to how leftand right images are combined into a 3D stereoscopic image, the 3Dstereoscopic imaging method is divided into a top-down method in whichleft and right images are disposed up and down in a frame, an L-to-R(left-to-right, side by side) method in which left and right images aredisposed left and right in a frame, a checker board method in whichfragments of left and right images are disposed in a tile form, aninterlaced method in which left and right images are alternatelydisposed by columns and rows, and a time sequential (or frame by frame)method in which left and right images are alternately displayed by time.

Also, as for a 3D thumbnail image, a left image thumbnail and a rightimage thumbnail are generated from a left image and a right image of theoriginal image frame, respectively, and then combined to generate asingle 3D thumbnail image. In general, thumbnail refers to a reducedimage or a reduced still image. The thusly generated left imagethumbnail and the right image thumbnail are displayed with a horizontaldistance difference therebetween by a depth corresponding to thedisparity between the left image and the right image on the screen,providing a stereoscopic space sense.

A left image and a right image required for implementing a 3Dstereoscopic image may be displayed on the stereoscopic display unit bya stereoscopic processing unit. The stereoscopic processing unit mayreceive the 3D image (an image of a reference time point and an image ofan extension time point), and extract the left image and the rightimage, or may receive 2D images and change them into a left image and aright image.

The audio output module 152 may receive audio data from the wirelesscommunication unit 110 or output audio data stored in the memory 170during modes such as a signal reception mode, a call mode, a recordmode, a voice recognition mode, a broadcast reception mode, and thelike. The audio output module 152 may provide audible output related toa particular function (e.g., a call signal reception sound, a messagereception sound, etc.) performed by the HMD 100. The audio output module152 may also be implemented as a receiver, a speaker, a buzzer, or thelike.

A haptic module 153 can be configured to generate various tactileeffects that a user feels, perceives, or otherwise experiences. Atypical example of a tactile effect generated by the haptic module 153is vibration. The strength, pattern and the like of the vibrationgenerated by the haptic module 153 may be controlled by user selectionor setting by the controller 180. For example, the haptic module 153 mayoutput different vibrations in a combining manner or a sequentialmanner.

Besides vibration, the haptic module 153 can generate various othertactile effects, including an effect by stimulation such as a pinarrangement vertically moving to contact skin, a spray force or suctionforce of air through a jet orifice or a suction opening, a touch to theskin, a contact of an electrode, electrostatic force, an effect byreproducing the sense of cold and warmth using an element that canabsorb or generate heat, and the like.

The haptic module 153 may also be implemented to allow the user to feela tactile effect through a muscle sensation such as the user's head,face, fingers or arm, as well as transferring the tactile effect throughdirect contact. Two or more haptic modules 153 may be provided accordingto the particular configuration of the HMD 100.

An optical output module 154 may output a signal for indicating an eventgeneration using light of a light source of the HMD 100. Examples ofevents generated in the HMD 100 may include a message reception, a callsignal reception, a missed call, an alarm, a schedule notice, an emailreception, information reception through an application, an output of animage (image, video, etc.) and the like. That is, the optical outputmodule 154 may notify that the HMD 100 is performing a specificoperation (function) by the user.

A signal output by the optical output module 154 may be implemented insuch a manner that the HMD emits monochromatic light or light with aplurality of colors. The signal output may be terminated in response tothe HMD sensing a user's event confirmation, or an operation currentlyperformed in the HMD being ended.

The interface unit 160 serves as an interface for every external deviceto be connected with the electronic device 100. For example, theinterface unit 160 may receive data transmitted from an external device,receive power to transfer to elements and components within the HMD 100,or transmit internal data of the HMD 100 to such external device. Theinterface unit 160 may include wired or wireless headset ports, externalpower supply ports, wired or wireless data ports, memory card ports,ports for connecting a device having an identification module, audioinput/output (I/O) ports, video I/O ports, earphone ports, or the like.

The identification module may be a chip that stores various informationfor authenticating authority of using the HMD 100 and may include a useridentity module (UIM), a subscriber identity module (SIM), a universalsubscriber identity module (USIM), and the like. In addition, the devicehaving the identification module (also referred to herein as an“identifying device”) may take the form of a smart card. Accordingly,the identifying device can be connected with the terminal 100 via theinterface unit 160.

When the HMD 100 is connected with an external cradle, the interfaceunit 160 may serve as a passage to allow power from the cradle to besupplied to the HMD 100 or may serve as a passage to allow variouscommand signals input by the user from the cradle to be transferred tothe HMD therethrough. Various command signals or power input from thecradle may operate as signals for recognizing that the HMD 100 isproperly mounted on the cradle.

The memory 170 can store programs to support operations of thecontroller 180 and store input/output data (for example, phonebook,messages, still images, videos, etc.). The memory 170 may store datarelated to various patterns of vibrations and audio which are output inresponse to touch inputs on the touch screen.

The memory 170 may include one or more types of storage mediumsincluding a flash memory type, a hard disk type, a solid state disk(SSD) type, a silicon disk drive (SDD) type, a multimedia card microtype, a card-type memory (e.g., SD or DX memory, etc), a Random AccessMemory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory(ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM),a Programmable Read-Only memory (PROM), a magnetic memory, a magneticdisk, an optical disk, and the like. The HMD 100 may also be operated inrelation to a network storage device that performs the storage functionof the memory 170 over a network, such as the Internet.

Meanwhile, as aforementioned, the controller 180 may typically controloperations relating to application programs and the general operationsof the HMD 100. For example, the controller 180 may set or release alock state for restricting a user from inputting a control command withrespect to applications when a status of the HMD meets a presetcondition.

The controller 180 may also perform the controlling and processingassociated with voice calls, data communications, video calls, and thelike, or perform pattern recognition processing to recognize ahandwriting input or a picture drawing input performed on the touchscreen as characters or images, respectively. In addition, thecontroller 180 may control one or a combination of those components inorder to implement various exemplary embodiments disclosed herein on theHMD 100 according to the present invention.

The power supply unit 190 receives external power or provides internalpower and supply the appropriate power required for operating respectiveelements and components included in the mobile terminal 100 under thecontrol of the controller 180. The power supply unit 190 may include abattery, which is typically rechargeable or be detachably coupled to theHMD Body for charging.

The power supply unit 190 may include a connection port. The connectionport may be configured as one example of the interface unit 160 to whichan external charger for supplying power to recharge the battery iselectrically connected.

As another example, the power supply unit 190 may be configured torecharge the battery in a wireless manner without use of the connectionport. In this example, the power supply unit 190 can receive power,transferred from an external wireless power transmitter, using at leastone of an inductive coupling method which is based on magnetic inductionor a magnetic resonance coupling method which is based onelectromagnetic resonance.

Various embodiments described herein may be implemented in acomputer-readable medium, a machine-readable medium, or similar mediumusing, for example, software, hardware, or any combination thereof.

Referring to FIG. 2A, FIG. 2A is a conceptual view illustrating an HMDrelated to the present invention, viewed in one direction.

Referring to FIG. 2A, the HMD 100 according to the present invention maybe formed to be worn on a head portion of a human body (or a head, aface, etc.), and include a frame part (a case, a housing, a cover,etc.). The frame part may be formed of a flexible material to facilitatewearing. This drawing illustrates that the frame part includes a firstframe 101 and a second frame 102 which are made of different materials.

For example, the first frame 101 may serve to provide a space in whichat least one of the components described with reference to FIG. 1 can bedisposed, and the second frame 102 may serve to support (or fix) thefirst frame 101 to be mounted on the head portion of the human body.

The frame part may be referred to as a main body (or an HMD main body)or a body (or an HMD body). Here, the HMD main body (or the HMD body)may also be understood as a conception referring to as at least oneassembly.

The frame part is supported on the head portion, and defines a space formounting various components. As illustrated, electronic components suchas a camera 121, an output unit, a user input unit 123, a controller (orcontrol unit), a sensing unit, and the like may be mounted on the firstframe 101. Here, the display unit 151 may be formed to cover at leastone of left and right eyes of a user (or at least one of the left andright eyes of the user), and may be detachably provided.

The second frame 102 may be provided with an electronic component suchas the audio output module 152 or the like. However, the presentinvention is not limited to this, and the components described in FIG. 1and the components necessary for the HMD may be variously arranged inthe first frame 101 and the second frame 102 according to a userselection.

The controller 180 (see FIG. 1) is configured to control variouselectronic components included in the HMD 100. The processor 180 may beunderstood as a component corresponding to the controller 180 describedin FIG. 1.

The display unit 151 is mounted on the frame part and outputs screeninformation (e.g., image, moving picture, video, etc.) to the front ofthe user's eyes. When the user wears the HMD 100, the display unit 151may be arranged to correspond to at least one of the left eye and theright eye so that screen information can be displayed in front of theuser's eyes. This drawing illustrates that the display unit 151 ispositioned so as to cover both the left and right eyes so that an imagecan be output toward both the left and right eyes of the user.

Also, the display unit 151 may project an image to the user's eyes usinga prism. Further, the prism may be formed to be transmissive so that theuser can view the projected image and a general front view (a rangeviewed by the user through the eyes) together.

As described above, the image output through the display unit 151 may beviewed in a manner of overlapping the general view. The HMD 100 mayprovide Augmented Reality (AR), in which a virtual image is superimposedon a real image or a background to be viewed as a single image, by usingthe characteristics of the display.

In addition, the display unit 151 of the HMD according to the presentinvention may be located inside the main body. Specifically, when theHMD is worn on the user's head, the display unit 151 may be disposed ata position facing the user's eyes inside the HMD.

The camera 121 is disposed adjacent to at least one of the left eye andthe right eye, to capture (take, photograph) an image of a front side.Since the camera 121 is disposed adjacent to the eye to face forward,the camera 121 may acquire a scene viewed by the user as an image.

In addition, the HMD according to an embodiment of the present inventioncaptures a preset object disposed outside by using the camera 121, anddetects a movement (motion) of the main body of the HMD using a capturedimage of the preset object.

For example, the camera 121 may be a camera having a preset field ofview (or a viewing angle). In addition, the camera 121 may capture apreset object included in a specific area located outside the HMD.

Here, the preset object may refer to all kinds of objects to beidentified, which form specific information. That is, the preset objectmay include characters, symbols, images, and objects forming thespecific information. For example, the preset object may be a pluralityof light emitting elements arranged to form a specific pattern. That is,the plurality of light emitting elements may form specific patterninformation according to spaced intervals therebetween or emitted light.As another example, the preset object may be a code (for example, a QRcode) having specific meaning or an image, a symbol, and the like, whichform a specific pattern.

The controller 180 may capture the preset object included in thespecific area using the camera 121, and analyze an image of the presetobject included in the captured image to determine the movement of themain body. For example, the controller 180 may determine the movement ofthe main body based on a degree that the image of the preset objectincluded in the captured image is deformed. Also, the controller 180 maycontrol an output state of the display unit 151 based on the movement ofthe main body. Detailed description related to this will be given laterwith reference to FIG. 4C.

This drawing illustrates that one camera 121 is provided, but thepresent invention is not limited thereto. The camera 121 may be providedin plurality to acquire a stereoscopic image.

The HMD 100 may include a user input unit 123 configured to receive acontrol command. The user input unit 123 may employ any method so longas it can be operated by the user in a tactile manner, such as touchingor pushing. This drawing illustrates that the frame part is providedwith the user input part 123 employing pushing and touching inputmanners.

In addition, the HMD 100 may be provided with a microphone (notillustrated) for receiving sound and processing the sound into anelectric audio data, and an audio output module 152 for outputtingsound. The audio output module 152 may be configured to transmit soundin a general sound output manner or a bone-conduction mode. In casewhere the audio output module 152 is implemented in the bone-conductionmanner, when the user wears the HMD 100, the audio output module 152 isbrought into close contact with the head and vibrates a skull totransmit sound.

Meanwhile, if the frame part including the first frame 101 and thesecond frame 102 is regarded as one HMD body (hereinafter, referred toas “main body”), the main body of the HMD related to the presentinvention may be configured in various forms. Specifically, the mainbody may include a plurality of surfaces 103 a, 103 b, and 103 c havingpreset angles. The plurality of surfaces indicates outer surfaces of themain body of the HMD 100. From this perspective, the plurality ofsurfaces may indicate the surfaces (outer surfaces, etc.) of the HMD100. Each of the plurality of surfaces 103 a, 103 b, and 103 c may beflat or curved.

FIG. 2A illustrates a main body in which the plurality of surfaces 103a, 103 b, and 103 c are perpendicular to one another. Hereinafter, forthe sake of convenience of explanation, an HMD formed so that aplurality of surfaces is perpendicular to one another will be describedas an example. However, the same/like description may be applicable toall types of HMDs including a plurality of surfaces having presetangles.

FIG. 2B is a conceptual view of an HMD according to another embodimentof the present invention, viewed in one direction.

The structure and configuration of the HMD illustrated in FIG. 2B may besubstantially the same as the structure and configuration of the HMDdescribed with reference to FIG. 2A. However, the preset object may bemounted on at least part of the plurality of surfaces 103 a, 103 b, and103 c forming the main body of the HMD 100 illustrated in FIG. 2B. Forexample, a plurality of light emitting units 200 a, 200 b, and 200 c foremitting preset light to outside may be disposed on at least part of theplurality of surfaces 103 a, 103 b, and 103 c. As another example, atleast part of the plurality of surfaces 103 a, 103 b, and 103 c may beprovided with arbitrary images and symbols to form specific information.

On the other hand, a camera for capturing a preset viewing angle range(or a preset range of field of view) may be located on the outside ofthe HMD. The controller 180 may receive a captured image from the cameralocated at the outside using the wireless communication unit. Inaddition, the controller 180 may determine a posture of the HMD 100 byanalyzing the captured image received from the external camera.

More specifically, when the main body having the preset object isincluded in the preset viewing angle range of the external camera, thecaptured image by the camera may include an image of the preset object.At this time, the controller 180 may determine the posture of the mainbody by analyzing a degree that the image of the preset object includedin the captured image is deformed.

Here, the posture of the HMD 100 may be understood as a conceptionincluding a placed state of the HMD (a worn state of the HMD on theuser's head), a tilted degree of the HMD, a direction that the HMDfaces, a position of the HMD, a rotation of the HMD, a movement of theHMD, and the like. In addition, determining the posture of the HMD 100in the present invention may include determining posture and movement ofthe user wearing the HMD 100.

When the posture of the HMD is determined as described above, thecontroller 180 may use the result to recognize the movement of the mainbody, and perform a control such as changing the output state of thedisplay unit based on the movement of the main body. Hereinafter, thiswill be described in detail with reference to FIG. 4B.

As described above, when the image of the preset object included in thecaptured image with respect to the preset object is changed, the HMDaccording to the present invention may recognize the movement of themain body by analyzing the image change of the preset object

When the movement of the main body is recognized, not only a movementthat the main body is tilted but also a movement that the main body ismoved in a horizontal direction without being tilted (e.g., a movementof the main body in parallel to left or right or forward or backward)may be precisely recognized.

In addition, the HMD according to the present invention may controlvarious functions by using the movement of the main body when themovement of the main body is recognized. For example, it may be possibleto control the output state of the display unit based on the movement ofthe main body, or perform a specific function based on a detection of aspecific movement of the main body. If screen information output on thedisplay unit is changed based on the movement of the main body, the userwearing the HMD according to the present invention may receive morerealistic screen information.

When the movement of the main body is recognized based on the imagechange of the preset object included in the captured image, the image ofthe preset object may deviate from the preset viewing angle range of thecamera due to the movement of the main body. In this case, the HMDaccording to the present invention may allow the user to recognize invarious ways that the main body is out of a moving area of the mainbody, in which the main body is to be moved for a function control.Hereinafter, a related control method will be described with referenceto the accompanying drawings.

FIG. 3 is a flowchart illustrating a control method according to thepresent invention, and FIGS. 4A, 4B, and 4C are representative viewsillustrating a method of controlling an HMD in accordance with oneembodiment of the present invention.

Referring to FIG. 3, screen information may be output on the displayunit of the HMD according to one embodiment of the present invention(S301).

As described above with reference to FIGS. 2A and 2B, the display unitmay be located at a position facing the user's eyes within the HMD, andmay be configured to output screen information.

The screen information may be an execution screen corresponding to afunction executable in the HMD. For example, the screen information mayinclude an execution screen of a specific application, image informationrelated to a stopped state, and image information related to acontinuous image that changes according to a lapse of time.

When the main body moves while the screen information is output on thedisplay unit 151, the controller 180 may change the screen informationbased on the movement of the main body.

More specifically, the HMD according to one embodiment of the presentinvention may acquire a captured image including an image of a presetobject, in order to recognize a movement of the main body. Here, thepreset object may refer to an object specified as an object to beanalyzed in the captured image with respect to the movement of the mainbody. In addition, the preset object may include all kinds of objects(characters, symbols, images, objects, etc.) to be identified that formspecific information, as described above with reference to FIG. 2A.

The captured image may be acquired from the camera that captures thepreset object. In addition, the captured image may be obtained indifferent ways depending on the position of the camera.

For example, when the camera is mounted on the main body and the presetobject is disposed outside the main body, the controller 180 maydirectly acquire the captured image from the camera mounted on the mainbody. Alternatively, when the camera is mounted at the outside of themain body and the preset object is mounted on at least part of the mainbody, the captured image may be obtained based on wireless communicationwith the external camera. As described above, the method of acquiringthe captured image may differ depending on the structure of the HMD.This will be described later with reference to FIGS. 4B and 4C.

In addition, the controller 180 may control the display unit to changethe screen information based on a detection of a deformation of an imageof a preset object in the captured image including the image of thepreset object (S302).

The image of the preset object may be, for example, a pattern imageconfigured to form specific information. That is, referring to FIG. 4A,the preset object may be mounted on the main body of the HMD 100according to one embodiment of the present invention, and the presetobject may be captured by an external camera 300. Here, the presetobject may be mounted to form a specific pattern on the main body.

Meanwhile, the external camera 300 may capture at least one subjectincluded in a preset viewing angle range 300 a. For example, when themain body of the HMD 100 is included in the viewing angle range 300 a,the captured image may include an image of the main body of the HMD 100.Similar to the structure of FIG. 4A, when the preset object is mountedon the main body of the HMD 100, the image of the preset object may beincluded in the captured image.

When the preset object is mounted on the main body of the HMD 100, forconvenience of explanation, the preset object is limited to a case ofbeing mounted on a front surface of the main body of the HMD 100.However, the preset object may be mounted not only on the front surfaceof the main body but also on each of the plurality of surfaces formingthe main body.

The controller 180 may detect a specific pattern image formed by thepreset object in the captured image. For example, as illustrated in afirst diagram of FIG. 4A, the controller 180 may detect a specificpattern image 10 a formed by the preset object from the captured image.

In this case, when the main body of the HMD 100 is moved as illustratedin a second diagram of FIG. 4A, the controller 180 may detect adeformation of the image 10 a in the captured image. The movement of themain body of the HMD 100 may be determined by analyzing a deformed image10 b detected from the captured image.

For example, when the deformed image 10 b is determined to be tilted ina predetermined direction (for example, to left) as compared with thespecific pattern image 10 a detected before the movement of the mainbody of the HMD 100, the controller 180 may determine that the main bodyhas been moved in an opposite direction to the predetermined direction.

The controller 180 may change the screen information based on themovement of the main body. For example, as illustrated in the firstdiagram of FIG. 4A, when the movement of the main body is detected whilespecific screen information is output on the display unit 151, thecontroller 180, as illustrated in a second diagram of FIG. 4B, maycontrol the display unit 151 such that at least part of the specificscreen information is moved to right.

Here, as illustrated in a third diagram of FIG. 4A, when the main bodymoves in a direction toward the external camera 300, the controller 180may detect an enlarged image 10 c of the deformed image 10 b from thecaptured image.

The controller 180 may then control the display unit 151 such that thespecific screen information output on the display unit 151 in the seconddiagram of FIG. 4A is enlarged to correspond to the enlarged image 10 c.

Similarly, in this state, when the main body moves away from theexternal camera 300, as illustrated in a fourth diagram of FIG. 4A, thecontroller 180 may detect a deformed image 10 d based on the movement ofthe main body from the captured image. The controller 180 may controlthe display unit 151 to output the specific screen information in areduced manner according to a degree that the deformed image 10 d isreduced from the enlarged image 10 c due to the movement of the mainbody.

In such manner, the HMD according to the present invention can determinethe movement of the main body by detecting the deformation of the imageof the preset object from the captured image, and change the outputstate of the display unit to correspond to the movement of the mainbody.

FIG. 4A has illustrated the embodiment in which the captured image isacquired from the camera mounted on the outside of the HMD, but thecamera may alternatively be mounted on the main body of the HMD, asillustrated in FIG. 2A. In this case, the captured image may be obtainedfrom the camera mounted on the main body, and the preset object may bedisposed outside the HMD.

Meanwhile, when at least part of the preset object deviates from apreset viewing angle range of the camera due to the movement of the mainbody, the controller 180 may output alert information indicating thedeviation.

More specifically, when at least part of the image is not detected inthe captured image, the controller 180 may control the display unit todisplay alert information related to the movement of the main bodytogether with the screen information (S303).

When the preset object is fully included in the preset viewing anglerange of the camera even if the main body moves, the controller maydetermine the movement of the main body based on the modification of theimage of the preset object included in the captured image.

However, when the preset object deviates from the preset viewing anglerange of the camera due to the movement of the main body, at least partof the image of the preset object may not be detected in the capturedimage. In this case, the controller 180 may analyze whether the entireimage is detected in the captured image, and determine whether thepreset object has deviated from the preset viewing angle range.

The controller 180 may provide alert information to the user to indicatethat the main body is moving to an area where the movement of the mainbody cannot be determined when at least part of the image is notdetected in the captured image. That is, the controller 180 may outputalert information on the display unit 151 to indicate that the main bodyhas moved out of the moving area of the main body in which the screeninformation output on the display unit 151 is controllable.

Here, the moving area may be determined based on the preset viewingangle range of the camera in which the preset object is captured.

The alert information may be various types of visual informationindicating a specific movement of the main body. This will be describedlater with reference to FIG. 5.

Meanwhile, as described above, the camera may be an external cameradisposed outside the HMD or a camera mounted on the HMD.

Referring to FIG. 4B, the camera for capturing the preset object may bean external camera 300 mounted at the outside, and the external camera300 may have a preset viewing angle range 300 a.

The external camera 300 may capture the preset object mounted on themain body 300 and the controller 180 may acquire a captured imageincluding an image of the preset object through wireless communicationwith the external camera 300.

When the captured image is acquired from the external camera 300, thecontroller 180, as illustrated in a first diagram of FIG. 4B, may detectan image 11 a, which is formed based on an arrangement of the presetobject, in the captured image.

In this case, when the main body of the HMD 100 moves as illustrated ina second diagram of FIG. 4B, the controller 180 may detect that theimage of the preset object is deformed in the captured image.

That is, the controller 180 may detect a deformed image 11 b of theimage 11 a and determine the movement of the main body using thedeformed image 11 b. The controller 180 may control the display unit 20to output screen information 21, a part of which has been moved to rightfrom previously-output screen information 20 to correspond to themovement of the main body (e.g., a case where the main body is rotatedclockwise by a predetermined range).

Also, at least part of the preset object may deviate from the presetviewing angle range 300 a due to the movement of the main body. In thiscase, the controller 180 may determine that the main body is out of themoving area of the main body by using at least part of the image of thepreset object that has not been detected in the captured image.

In addition, the controller 180 may control the display unit 151 tooutput alert information, which indicates that the movement of the mainbody is out of the moving area of the main body, together with thescreen information 21.

In this case, as illustrated in a second diagram of FIG. 4B, alertinformation 1 may be displayed along a periphery of the screeninformation 21 so as not to interfere with the output of the screeninformation 21. Also, the alert information 1 may be displayed in apartial area of the screen information 21 to correspond to a degree thatthe main body deviates from the preset viewing angle range 300 a.

On the other hand, a camera for capturing the preset object may bemounted on the main body of the HMD. That is, as illustrated in a firstdiagram of FIG. 4C, a camera 121 having a preset viewing angle range 121a may be mounted on a front surface of the main body, to capture apreset object existing outside the main body.

At this time, when the main body is partially rotated clockwise, theimage of the preset object may be deformed in the captured image. Thatis, the controller 180 may detect a deformed image 12 b, as illustratedin a second diagram of FIG. 4C, in the captured image.

The controller 180 may control the display unit 151 such that the screeninformation 20 is changed to screen information 22, in which a partialarea of the screen information 20 has moved to right, by reflecting themovement of the main body determined by the deformed image 12 b.

In addition, when at least part of the image is not detected in thecaptured image, the controller 180 may analyze the image 12 b of theremaining part of the image so as to determine the movement of the mainbody. The controller 180 may control the display unit 151 such thatalert information indicating the deviation of the main body out of themoving area of the main body is output on one area of the screeninformation 22 together with the screen information 22.

In this manner, the HMD according to the present invention may analyzethe change in the image of the preset object included in the capturedimage, recognize precisely various movements of the main body, andeffectively control screen information using the recognized movements.Accordingly, the user can move the HMD in various ways while wearing theHMD to control the screen information to match his/her intention.

In addition, the HMD according to the present invention may provide anotification to the user when at least part of the image of the presetobject deviates from the preset viewing angle range of the camera sothat the user can accurately determine the movement of the main body.That is, the present invention may set the moving area of the main body,in which the movement of the main body can be determined, based on theviewing angle range of the camera, and provide alert information to theuser when the main body is out of the moving area.

Accordingly, the user can control the screen information by moving themain body while recognizing a moved degree of the main body using thealert information, thereby preventing a drastic change of the screeninformation or a temporarily-stopped visual phenomenon.

The method of controlling the HMD according to the embodiment of thepresent invention, as illustrated in FIGS. 4B and 4C, may be implementedby both of the case where the camera is mounted on the main body of theHMD and the case where the camera is mounted at the outside of the HMD.However, the following description will focus on embodiments implementedbased on the structure of the HMD illustrated in FIG. 2C and the cameraposition in FIG. 4B.

In addition, the preset object may include all kinds of targets(objects) to be identified, which form specific information.Hereinafter, for convenience of explanation, the following descriptionwill mainly be given of an example in which the preset objectcorresponds to a plurality of light emitting elements each emitting apreset type light, with reference to the following appended drawings.

That is, the plurality of light emitting elements may be mounted on themain body of the HMD to form a specific pattern, and an image capturedby an external camera may include an optical image pattern formed bylight emitted by each of the plurality of light emitting elements.

In addition, the controller 180 may determine a movement of the mainbody using a number of optical images forming the optical image pattern.More specifically, when the plurality of light emitting elements are allincluded within a preset viewing angle range of the camera, thecontroller 180 may determine that the number of optical images formingthe optical image pattern formed by each of the plurality of lightemitting elements is a reference number.

In addition, when at least part of the plurality of light emittingelements deviates from the preset viewing angle range due to themovement of the main body, the controller 180 may determine that thenumber of optical images forming the optical image pattern included inthe captured image has been reduced to be smaller than the referencenumber.

In this case, the controller 180 may determine various movements of themain body based on a degree that the number of optical images formingthe optical image pattern becomes smaller than the reference number. Asa detailed example, the controller 180 may determine a moving speed ofthe main body based on a degree that the main body is out of the movingarea or a degree that the number of optical images is reduced for apreset period of time.

The alert information may be output on the display unit 151 as variousvisual effects which can indicate that the main body is out of themoving area of the main body. In addition, the alert information may beoutput as a specific visual effect on at least part of the screeninformation so as not to affect the user receiving the screeninformation.

FIGS. 5A, 5B and 5C are views illustrating embodiments related tovarious output forms of alert information.

First, referring to FIG. 5A, when the main body is out of the movingarea of the main body while the screen information 20 is output on thedisplay unit 151, the controller 180 may recognize a degree that themain body is out of the moving area. For example, the controller 180 maydetermine a deviated degree of the main body from the moving area, basedon a degree that the number of optical images forming the optical imagepattern is detected to be smaller than the reference number in thecaptured image.

The controller 180 may specify a part of the screen information 20 tocorrespond to the deviated degree of the main body from the moving area,and output alert information 3 on the specified part. That is, asillustrated in FIG. 5A, the alert information may be output by a visualeffect 3 in which a part of the screen information 20 is darkened topreset first brightness.

Alternatively, as illustrated in FIG. 5B, the alert information may bedisplayed by a visual effect 4 in which a part of the screen informationspecified to correspond to the deviated degree of the main body from themoving area is brightened to preset second brightness.

Also, as illustrated in FIG. 5C, the controller 180 may display thealert information using a visual effect 5 such as pixelation on thespecified part of the screen information.

Although not illustrated, the alert information may also be displayed bya visual effect such as generating a border on a part of a periphery ofthe screen information, or blurring a part of the screen information.

In this way, since the alert information is output on a part of thescreen information by the changed brightness or the specific visualeffect while outputting the screen information, the user can recognizethe movement of the main body while receiving the screen informationwithout a limit.

Meanwhile, the controller 180 may control the output state of thedisplay unit in various ways using a speed at which the main body movesout of the moving area. For example, when the main body moves out of themoving area at a first speed, the alert information may overlap thescreen information corresponding to the first speed. In addition, whenthe main body completely deviates from the moving area at a second speedhigher than the first speed, the controller 180 may overlap the alertinformation with the entire screen information.

More specifically, to recognize the moving speed of the main body, thecontroller 180 may determine a degree of change of at least part of theimage, which has not been detected in the captured image for a presettime. That is, when the at least part of the image has changed to afirst degree of change for a preset reference time, the controller 180may control the display unit 151 to overlap the alert information withthe part of the screen information so that the alert informationcorresponds to the first degree of change. When the at least part of theimage has changed to a second degree of change, the controller 180 maycontrol the display unit 151 to overlap the alert information with theentire screen information.

When the preset object is the plurality of light emitting elements, thecontroller 180 may determine the moving speed of the main body based ona degree of change of the number of optical images configuring theoptical image pattern, detected for the preset reference time. In thiscase, the controller 180 may control the output state of the displayunit 151 using a degree that the number of optical images detected forthe preset time is reduced from the reference number. This will bedescribed with reference to FIGS. 6A and 6B.

FIGS. 6A and 6B are views illustrating an embodiment for controlling anoutput of alert information based on a moving speed of a main body.

First, referring to FIG. 6A, when the main body moves in a state inwhich screen information 23 is output on the display unit 151, asillustrated in a second diagram of FIG. 6A, the controller 180 mayoutput alert information 6 a to overlap a part of the screen information23.

More specifically, the controller 180 may determine the degree to whichthe number of optical images constituting the optical image pattern isreduced from the reference number for a preset time. For example, whenthe number of optical images constituting the optical image pattern isdetected to be smaller than the reference number by a specific numberfor the preset time, the controller 180 may determine that the main bodymoves at a first speed.

The controller 180 may control the display unit 151 such that the alertinformation overlaps the screen information to correspond to the firstspeed. For example, the controller 180 may calculate a ratio of thespecific number to the reference number, and specify one area, whichcorresponds to the calculated ratio in the screen information. Also, thecontroller 180 may control the display unit 151 such that the alertinformation gradually overlaps at the first speed from an edge of thescreen information to the specified one area for the preset time.

When the main body continuously moves at the first speed while the alertinformation 6 a overlaps the one area of the screen information 23, thecontroller 180 may include specific information in the alertinformation.

More specifically, when at least part of the image which is not detectedin the captured image is changed to a first degree of change, thecontroller 180 may control the display unit 151 such that the alertinformation overlaps a part of the screen information. When a rate thatthe alert information overlaps the screen information corresponds to apreset ratio, the controller 180 may control the display unit 151 todisplay the specific information in the alert information.

For example, after the alert information is displayed, when the numberof optical images constituting the optical image pattern is detected asmany as a preset threshold coefficient based on the movement of the mainbody, the controller 180 may further provide the specific information inthe alert information.

That is, as illustrated in a third diagram of FIG. 6A, the alertinformation may include as the specific information 6 b menu informationfor changing settings in relation to a function corresponding to thescreen information 23.

In this way, since the controller 180 displays the alert informationcorresponding to the moving speed of the main body, the user can movethe main body while recognizing the movement of the main body. Inaddition, when a continuous movement of the main body is detected afterthe output of the alert information, specific information may beadditionally provided in the alert information so as to match theintention of the user who continuously moves the main body.

On the other hand, the user may move the main body at a second speedthat is faster than the first speed. That is, as illustrated in a firstdiagram of FIG. 6B, in a state where the screen information 23 is outputon the display unit 151, the main body may move at the second speedwithin the preset time.

For example, the second speed may be a speed at which the main bodymoves so as to completely deviate from the moving area of the main bodyfor the preset time. That is, in case where the entire image of thepreset object is not detected for the preset reference time, forexample, when the number of optical images constituting the opticalimage pattern is detected as being reduced by the reference number, thecontroller 180 may determine that the main body has moved at the secondspeed.

The controller 180, as illustrated in a second diagram of FIG. 6B, maycontrol the display unit 151 such that alert information 7 overlaps theentire screen information 23. In addition, the alert information 7 maycontrol the display unit 151 to output information indicating that themain body has moved out of the moving area.

In this way, when the main body completely deviates from the moving areawithin the preset reference time, the controller 180 may determine thatthe user does not intend to use the screen information, and then controlthe alert information to overlap the entire screen information.

The user wearing the HMD can not only unintentionally move the main bodyaway from the moving area of the main body, but also intentionally movethe main body away from the moving area of the main body. Therefore, thepresent invention can control the output state of the display unit basedon the moving speed of the main body, thereby performing more assortedcontrol for the user's intention.

As described above with reference to FIG. 6A, when the main bodycontinuously moves at the first speed while the alert information isdisplayed, specific information may be additionally output in the alertinformation. Hereinafter, an embodiment related to the specificinformation will be described with reference to FIGS. 7A and 7B.

FIGS. 7A and 7B are views illustrating an embodiment related to anexecution of a function related to the specific information when themain body moves in a state where the specific information isadditionally displayed in alert information.

First, referring to FIG. 7A, when a part of the main body is out of themoving area in a state where screen information 23 is output on thedisplay unit 151, the controller 180 may output alert information on thescreen information 23.

More specifically, the controller 180 may calculate a moving speed ofthe main body based on a degree of change of at least part of the imagewhich is not detected from the captured image received from the externalcamera 300 for a preset reference time. The controller 180 may controlthe display unit 230 so that the alert information 6 a overlaps a partof the screen information 23 to correspond to a degree of change (afirst degree of change) of the at least part of the image.

In addition, when an overlap ratio of the alert information 6 a with thescreen information 23 corresponds to a specific ratio, specificinformation may be displayed in the alert information 6 a. For example,when the preset object is the plurality of light emitting elements, thecontroller 180 may detect a change in the number of optical imagesconstituting the optical image pattern in the captured image.

When the number of optical images constituting the optical image patternis detected as may as a preset threshold number based on the movement ofthe main body, the controller 180 may display specific information inthe alert information. For example, as illustrated in a second diagramof FIG. 7A, the specific information may be recommendation information 6c for another function different from a function corresponding to thescreen information 23.

When the entire image is not detected in the captured image based on themovement of the main body in a state where the recommendationinformation 6 c is output on the alert information (when the opticalimages constituting the optical image pattern are not detected any morein case where the preset object is the plurality of light emittingelements), the controller may determine that the main body hascompletely deviated from the moving area.

And, as illustrated in a third diagram of FIG. 7A, the controller 180may execute the another function included in the recommendationinformation 6 c, and control the display unit 151 to output an executionscreen 24 in response to the execution of the another function. In thiscase, the controller 180 may terminate the execution of the functioncorresponding to the screen information 23, in response to the executionof the another function.

Although not illustrated, when the another function is a function thatis controllable based on the movement of the main body, the controller180 may provide the user with information recommending the movement ofthe main body such that the main body is located in the moving area. Inthis case, the controller 180 may temporarily suspend the execution ofthe another function until the main body is located in the moving area.

Accordingly, the user can be provided with information of anotherfunction executable while controlling the screen information with movingthe main body, and also provided with convenience in easily executingthe another function by a continuous movement of the main body.

On the other hand, when a preset gesture is detected by the sensing unitwhile the recommendation information is provided, the controller 180 mayexecute the another function included in the recommendation informationwhile executing the function corresponding to the screen information.

That is, as illustrated in a second diagram of FIG. 7B, in a state wherethe recommendation information 6 c related to the another functiondifferent from the function corresponding to the screen information 23is output in the alert information, the user's preset gesture may bedetected by the sensing unit. For example, the preset gesture, asillustrated in a second diagram of FIG. 7B, may be a gesture that theuser wearing the main body nods the head.

With no limit to this, the preset gesture may be defined as variousgestures such as a gesture of tilting the main body in a specificdirection, a gesture of shaking the main body in left and rightdirections by a preset number of times, a blinking gesture, and thelike.

When the preset gesture is detected, the controller 180 may execute theanother function included in the recommendation information whilemaintaining the execution of the function corresponding to the screeninformation. The controller 180, as illustrated in a third diagram ofFIG. 7B, may then control the display unit 151 such that the screeninformation 23 is changed to an execution screen 24 according to theexecution of the another function.

That is, the screen information 23 may be temporarily restricted frombeing output on the display unit 151 based on the execution of theanother function. In this state, when the main body moves to be locatedin the moving area, the controller 180 may detect the entire image inthe captured image.

For example, when the preset object is the plurality of light emittingelements, the controller 180 may detect a change in the number ofoptical images constituting the optical image pattern in the capturedimage. When the number of optical images constituting the optical imagepattern is detected as the reference number in the state where theexecution screen 24 is output according to the execution of the anotherfunction, the controller 180 may control the display unit 151 to changethe execution screen 24 back to the screen information 23. Also, whenthe screen information 23 is output on the display unit 151 again, thecontroller 180 may terminate the execution of the another function.

When the screen information is image information for a continuous imagethat changes according to a lapse of time, the controller 180 mayspecify an image output on the display unit 151 at a time point when theanother function is executed. Then, after temporarily suspending theimage information in the specified image, the image information may bechanged to the execution screen of the another function. Thereafter,when the execution of the another function is terminated, the controllermay control the display unit 151 to output the specified image includedin the image information.

In this way, in a state where the recommendation information isprovided, the controller 180 may simultaneously execute the anotherfunction and the function corresponding to the screen information, inresponse to the detection of the preset gesture. Although notillustrated, the controller 180 may temporarily execute the anotherfunction, in response to a specific voice input or touch input beingapplied by a user, as well as the detection of the preset gesture.

Therefore, when the user wants to temporarily use the another function(for example, a function of confirming a new event generated in theHMD), the user can execute the another function simply by applying aspecific gesture.

On the other hand, when the execution of the function corresponding tothe screen information is maintained together with the execution of theanother function, the controller 180 may output the execution screen ofthe another function and the screen information simultaneously on thedisplay unit 151.

FIGS. 8A and 8B are views illustrating an embodiment related tomultitasking of a function corresponding to screen information and afunction included in recommendation information.

First, referring to FIG. 8A, alert information may be displayed on thedisplay unit 151 together with screen information 23, and the alertinformation may include recommendation information 6 d for anotherfunction different from a function corresponding to the screeninformation 23.

At this time, when a user's preset gesture is sensed by the sensingunit, the controller 180 may execute the another function included inthe recommendation information 6 d. At this time, the controller 180 maydivide the display unit 151 into a plurality of areas, and, asillustrated in a second diagram of FIG. 8A, control the display unit 151to output the screen information 23 and an execution screen 24 of theanother function in the plurality of areas, respectively.

In this manner, when the execution of the function corresponding to thescreen information is maintained together with the execution of theanother function, the output of the screen information can be maintainedso that the user can simultaneously receive the screen information andthe execution screen of the another function.

In addition, in the state where the screen information and the executionscreen of the another function are simultaneously output, the controller180 may control the screen information or the execution screen of theanother function based on the user's input or gesture.

The controller 180 may specify a control area to be controlled based onthe user's input or gesture among the plurality of areas of the displayunit 151. Further, the display unit 151 may be controlled such that thespecified control area is distinguishably output. That is, asillustrated in a first diagram of FIG. 8B, the control area to becontrolled based on the user's input or gesture may be one area 24 aoutput on the execution screen 24 of the another function.

In this manner, when a user's specific gesture (for example, a gestureof nodding the head) is detected in a state where the control area 24 ahas been specified, the controller 180 may control the execution screen24 based on the specific gesture. That is, as illustrated in a seconddiagram of FIG. 8B, an execution screen 24 b may be output on thespecified one area 24 a based on the specific gesture.

Also, the controller 180 may change the control area, in response to apreset user gesture being applied after the one area 24 a is specified.For example, when a user gesture of blinking one eye is detected, thecontroller 180 may change the one area 23 a located at a positioncorresponding to the blinking eye to the control area.

In this way, when the control area is changed, the controller 180 mayoutput a specific visual effect along a periphery of the changed controlarea 23 a so that the changed control area 23 a is distinguishable.

Accordingly, the user can execute another function different from apreviously-output function corresponding to screen information, togetherwith the function corresponding to the screen information. In addition,a screen corresponding to each of different functions can be providedusing each of the plurality of divided areas of the display unit 151,and only a screen output on one area can selectively be controlled byapplying various gestures.

On the other hand, as described above with reference to FIG. 6B, whenthe main body completely moves out of the moving area while moving atthe second speed, the controller 180 may control the display unit 151 tooverlap the alert information with the entire screen information.

The controller 180 may control the output state of the display unit 151based on an additional movement of the main body in the state where thealert information overlaps the entire screen information.

FIGS. 9A and 9B are views illustrating an embodiment of controlling theoutput state of the display unit based on the movement of the main bodyin the state where the alert information overlaps the entire screeninformation.

Referring to FIG. 9A, the controller 180 may determine a moving speed ofthe main body, based on a degree of change of at least part of the imagein the captured image for a preset reference time in a state where thescreen information 23 is output on the display unit 151. When the entireimage is not detected for the preset reference time (when the degree ofchange of the at least part of the image is a second degree of change),the controller 180 may determine that the main body is completely out ofthe moving area of the main body.

For example, when the preset object is the plurality of light emittingelements, the controller 180 may determine the moving speed of the mainbody based on a degree of change of the number of optical imagesconstituting the optical image pattern, detected in the captured image,for the preset reference time.

When the number of optical images constituting the optical image patternis reduced by the reference number for the preset time, the controller180 may determine that the main body has completely moved out of themoving area at the second speed (see FIG. 6B). As illustrated in asecond diagram of FIG. 9A, alert information 7 may overlap the entirescreen information 23. In addition, the alert information 7 mayadditionally include information to indicate that the main body is outof the moving area.

When the alert information overlaps the entire screen information, thecontroller 180 may control the display unit 151 to terminate the outputof the screen information.

In addition, the controller 180 may analyze using the captured imagewhether the main body moves for a preset time from the termination ofthe output of the screen information. For example, when the number ofoptical images constituting the optical image pattern is detected as thereference number within the preset time, the controller 180 maydetermine that the main body is located within the moving area.

In this case, the controller 180, as illustrated in a third diagram ofFIG. 9A, may control the display unit 151 to display the screeninformation 23 again.

Alternatively, the main body may not be located in the moving areawithin a preset time or may be located in the moving area after a lapseof the preset time, from the termination of the output of the screeninformation 23.

For example, as illustrated in a second diagram of FIG. 9B, after thealert information 7 is output on the entire area of the display unit151, the main body may not be located within the moving area while thepreset time elapses.

In this case, the controller 180 may terminate the execution of thefunction corresponding to the screen information at the time when thepreset time has elapsed.

That is, although the main body moves completely out of the moving area,when the main body moves back into the moving area within a preset time,the controller 180 may determine that the user intends to continuouslycontrol the screen information. Accordingly, the controller 180 maytemporarily terminate the output of the screen information, and thenoutput the screen information again on the display unit so that thescreen information can be additionally controlled when the main bodymoves into the moving area again.

On the other hand, when the main body is not located within the movingarea even after the lapse of the preset time in the state where the mainbody is completely out of the moving area, the user may not intend tocontinuously control the screen information. Therefore, in this case,the controller 180 may terminate the executed function corresponding tothe screen information, thereby saving power.

Meanwhile, an output position of the alert information on the displayunit 151 may vary according to a direction in which the main body movesout of the moving area. More specifically, when a specific part of theimage of the preset object included in the captured image is notdetected, the controller 180 may determine a position of the specificpart of the image. Also, the controller 180 may control the display unit151 to display the alert information at a position on the display unit,which corresponds to the position of the specific part of the image.Hereinafter, this will be described with reference to FIG. 10.

FIG. 10 is a view illustrating an embodiment in which an output positionof alert information is changed based on a direction in which the mainbody moves out of a moving area.

Referring to a first diagram of FIG. 10, screen information 20 is outputon the display unit 151 and the main body of the HMD 100 is locatedwithin a preset viewing angle range 300 a of the external camera 300. Atthis time, as the main body is rotated clockwise, a part of the mainbody may deviate from the preset viewing angle range 300 a.

When the preset object is a plurality of light emitting elements, thecontroller 180 controls the display unit 151 5o output changed screeninformation 20 a based on a number of optical images constituting anoptical image pattern, detected in a captured image obtained from theexternal camera 300.

Also, the controller 180 may analyze the captured image to specify onearea where the optical image is not detected in the optical imagepattern. An output position of the alert information may be determinedbased on the specified one area of the optical image pattern. That is,referring to a second diagram of FIG. 10, the controller 180 may controlthe display unit 151 to display the alert information 9 a at a position,which corresponds to the specified one area of the optical image patternon the display unit 151.

When the main body moves back into the preset viewing angle area 300 aof the external camera 300 while the alert information 9 a is output, asillustrated in FIG. 10, the controller 180 may control the display unit151 to output the screen information 20. In addition, when the main bodyis located in the preset viewing angle range 300 a, the output of thealert information 9 a may be terminated.

When a part of the main body is out of the viewing angle range 300 a dueto a counterclockwise rotation of the main body, the controller 180 mayspecify another area, in which the optical image is not detected in theoptical image pattern. Also, as illustrated in a fourth diagram of FIG.10, the controller 180 may control the display unit 151 to output thealert information 9 b at a position, which corresponds to the anotherone area of the optical image pattern on the display unit 151.

Therefore, the user can easily recognize a direction in which the mainbody deviates from the moving area of the main body, based on the outputposition of the alert information on the display unit.

On the other hand, the plurality of light emitting elements mounted onthe main body of the HMD may be obscured by another subject. Forexample, when the user wearing the HMD touches the main body of the HMD,at least part of the plurality of light emitting elements may beobscured by the user's hand. In this case, the controller 180 may informthe user that the at least part of the plurality of light emittingelements is obscured by the user's hand. That is, when a movement of thespecific object with respect to the main body is detected in thecaptured image obtained by an external camera, the controller 180 mayprovide the user with notification information related to presence ofthe specific subject.

FIG. 11 is a view illustrating an embodiment of outputting informationinforming a movement of a specific subject (object) with respect to themain body.

As illustrated in a first diagram of FIG. 11, in a state where screeninformation 20 is output on the display unit 151, at least part of themain body may be obscured by a specific subject (e.g., a user's hand),as illustrated in a second diagram of FIG. 11. When the at least part ofthe main body is obscured, the controller 180 may recognize the presenceof the specific subject with respect to the main body using the capturedimage.

More specifically, when the specific subject moves with respect to themain body, capturing of at least part of the plurality of light emittingelements by the external camera 300 may be restricted. In this case, adetection of at least part of the optical images constituting theoptical image pattern from the captured image may be restricted.

The controller 180 may recognize the detection-restricted at least partof the optical images constituting the optical image pattern, by usingthe captured image. The controller 180 may recognize a position of theat least part of the light emitting elements, which corresponds to theat least part of the optical images, among the plurality of lightemitting elements mounted on the main body. The controller 180 maycontrol the display unit 151 to output notification informationindicating the movement of the specific subject to a positioncorresponding to the position of the at least part of the light emittingelements.

That is, as illustrated in the second diagram of FIG. 11, the controller180 may recognize a position of a part of the light emitting elements,which is obscured by the user's hand, among the plurality of lightemitting elements, and output notification information 9 c to a positionon the display unit 151 corresponding to the position of the part of thelight emitting elements.

Also, as illustrated in a third diagram of FIG. 11, when an obscuredarea of the main body is changed due to the movement of the specificsubject with respect to the main body, the controller 180 may change theoutput position of notification information 9 d to a positioncorresponding to the changed area.

Therefore, the notification information can be provided even though theuser unintentionally obscures at least part of the plurality of lightemitting elements, which may allow the user to recognize the obscuredarea.

According to the present invention, a movement of a main body of an HMDcan be recognized by using a captured image with respect to a presetobject, thereby accurately detecting various movements of the main body,and controlling various functions using the detected movements.

In addition, alert information can be output when capturing of at leastpart of the preset object is failed due to the movement of the mainbody, which may allow the user to recognize a moved state of the mainbody.

The present invention can be implemented as computer-readable codes in aprogram-recorded medium. The computer-readable medium may include alltypes of recording devices each storing data readable by a computersystem. Examples of such computer-readable media may include hard diskdrive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM,CD-ROM, magnetic tape, floppy disk, optical data storage element and thelike. Also, the computer-readable medium may also be implemented as aformat of carrier wave (e.g., transmission via an Internet). Thecomputer may include the controller 180 of the HMD. Therefore, it shouldalso be understood that the above-described embodiments are not limitedby any of the details of the foregoing description, unless otherwisespecified, but rather should be construed broadly within its scope asdefined in the appended claims, and therefore all changes andmodifications that fall within the metes and bounds of the claims, orequivalents of such metes and bounds are therefore intended to beembraced by the appended claims.

What is claimed is:
 1. A head mounted display (HMD), comprising: a mainbody formed to be wearable on a head portion of a user; a display unitmounted on one area of the main body and configured to output screeninformation; and a controller configured to control the display unit tochange the screen information based on a detection of deformation of animage of the preset object in a captured image including the image ofthe preset object, wherein the controller controls the display unit sothat alert information related to the movement of the main body isoutput together with the screen information when at least part of theimage is not detected in the captured image.
 2. The display of claim 1,further comprising a camera mounted on the main body, wherein the presetobject is disposed outside the head mounted display, and wherein thecontroller acquires the captured image including the image of the presetobject from the camera.
 3. The display of claim 1, further comprising awireless communication unit, wherein the preset object is mounted on atleast part of the main body, and wherein the controller controls thewireless communication unit to receive a captured image including theimage of the preset object from a camera located outside the headmounted display.
 4. The display of claim 1, wherein the preset object isa plurality of light emitting elements each emitting preset light. 5.The display of claim 1, wherein the controller controls the display unitto change an output of the alert information based on a moving speed ofthe main body determined from the captured image.
 6. The display ofclaim 5, wherein the controller determines the moving speed of the mainbody using a degree of change of at least part of the image for a presetreference time when the at least part of the image is not detected inthe captured image.
 7. The display of claim 6, wherein the controllercontrols the display unit such that the alert information overlaps apart of the screen information when the degree of change of the at leastpart of the image is a first degree of change, and controls the displayunit such that the alert information overlaps the entire screeninformation when the degree of change of the at least part of the imageis a second degree of change.
 8. The display of claim 7, wherein thecontroller controls the display unit such that the alert informationoverlaps the screen information to correspond to the first degree ofchange when the degree of change of the at least part is the firstdegree of change, and controls the display unit to displayrecommendation information for another function, different from afunction corresponding to the screen information, in the alertinformation when an overlap rate of the alert information with thescreen information corresponds to a preset specific rate.
 9. The displayof claim 8, wherein the controller executes the another functionincluded in the recommendation information when the entire image is notdetected in the captured image in a state where the recommendationinformation for the another function is displayed.
 10. The display ofclaim 9, wherein the controller terminates an execution of the functioncorresponding to the screen information when the another function isexecuted, and controls the display unit to output an execution screenaccording to the execution of the another function.
 11. The display ofclaim 8, further comprising a sensing unit configured to sense a presetgesture, wherein the controller executes the another function whilemaintaining the execution of the function corresponding to the screeninformation when the preset gesture is detected by the sensing unit in astate that the recommendation information is output, and controls thedisplay unit such that an execution screen according to the execution ofthe another function is output together with the screen information. 12.The display of claim 8, further comprising a sensing unit configured tosense a preset gesture, wherein the controller executes the anotherfunction included in the recommendation information when the presetgesture is detected by the sensing unit in a state that therecommendation information is output, and controls the display unit tochange the screen information to the execution screen according to theexecution of the another function.
 13. The display of claim 12, whereinthe controller controls the display unit to change the execution screenaccording to the execution of the another function back to the screeninformation when the entire image is detected in the captured image in astate that the screen information has changed to the execution screenaccording to the execution of the another function.
 14. The display ofclaim 7, wherein the controller, when the degree of change of the atleast part of the image is the second degree of change, controls thedisplay unit so that the alert information includes informationindicating that the main body is out of the moving area of the main bodyin which the main body is allowed to control the screen information. 15.The display of claim 1, wherein the controller, when a specific part ofthe image is not detected in the captured image, controls the displayunit such that the alert information is output at a position, whichcorresponds to a position of the specific part with respect to the imageon the display unit.
 16. The display of claim 3, wherein the controller,when a movement of a specific subject with respect to the main body isdetected from the captured image and at least part of the image is notdetected based on the movement of the specific subject, controls thedisplay unit such that the alert information includes notificationinformation indicating presence of the specific subject.
 17. A methodfor controlling a head mounted display (HMD), the method comprising:outputting screen information on a display unit mounted on one area ofthe main body; detecting deformation of an image of a preset object in acaptured image including the image of the preset object; changing thescreen information based on the detection of the deformation of theimage; and outputting alert information related to a movement of themain body together with the screen information when at least part of theimage is not detected in the captured image.
 18. The method of claim 17,wherein the preset object is disposed outside the head mounted display,and wherein the captured image including the image of the preset objectis acquired from a camera mounted on the main body.
 19. The method ofclaim 17, wherein the preset object is mounted on at least part of themain body, and wherein the captured image including the image of thepreset object is received from a camera located outside the head mounteddisplay through a wireless communication unit performing wirelesscommunication with the camera.
 20. The method of claim 17, wherein thealert information is differently displayed based on a degree of changeof the at least part of the image for a preset reference time when theat least part of the image is not detected in the captured image.